Multi-phase array redistribution: modeling and evaluation

s t lcm lcm*2 lcm*4 gcd gcd/2 gcd/4 s t lcm lcm*2 lcm*4 gcd gcd/2 gcd/4 Table 1: Execution times (ms) for cyclic(s) to cyclic(t) redistribution on 32 processors. other block sizes t. Fig. 3 shows the total times in milliseconds for a cyclic(192) to cyclic(8) redistribution on 32 processors for increasing data sizes. This redistribution corresponds to the cyclic(Y t) to cyclic(t) case with Y = 24. The two-phase redistribution is: cyclic(192) ! cyclic(48) ! cyclic(8). The two-phase strategy performs better than the single-phase strategy up to a data size of approximately 90K. Note that the cross over between the single-phase and two-phase redistribution occurs at a lower data size for the case when Y = 24 than for the case when Y = 30. This behavior can be explained by noting that the reduction in the number of message startups with the two-phase redistribution is greater for Y = 30. Similar patterns were observed for the cyclic(s) to cyclic(Y s) redistribution. We now evaluate the effect of choice of common multiples and divisors on the general cyclic(s) to cyclic(t) redistribution. Table. 1 shows the two-stage redistribution times for various source and target block sizes. The redistribution times for various multiples and divisors are presented. It can be observed that the two-stage redistributions using the lcm and the gcd as intermediate block sizes have nearly equal execution times and perform better than the two-stage redistributions using a common multiple greater than the lcm and a common divisor smaller than the gcd. 7 Conclusion We have presented a multi-phase approach for performing communication efficient data redistribution for block-cyclically distributed arrays. We have developed precise closed form expressions for the send and receive processor and data index sets for two special cases where the block size of the source block-cyclic distribution is a multiple of the block size of the target block-cyclic distribution and vice versa. These closed forms facilitate the development of a distributed scheduling algorithm for performing the all-to-many personalized communication, and the devel-opement of a communication cost model for array redistribution. Based on this model, we demonstrate that the use of the multi-phase redistribution strategy can reduce the total cost for array redistribution. Performance results on the Cray T3D show that the multi-phase strategy can improve performance over the single phase strategy for array redistribution. The multi-phase strategy is being extended to handle array redistribution of multi-dimensional arrays. …